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TITLE OF THE INVENTION 



Method and Apparatus for Coding Live Images in Microscopy 



CROSS REFERENCE TO RELATED APPLICATIONS 



5 [0001] This appHcation claims priority of the German patent application DE 100 26 
392.5 filed May 27, 2000 which is incorporated by reference herein. 



[0002] The invention concerns a method for coding live images in microscopy. In 
10 particular, the invention concems a method in which the microscope images that are 
recorded are partially coded. Displacement of the X-Y stage induces an offset of a 
portion of the previously recorded image; it is sufficient if only the new image portion is 
coded and transmitted. 

[0003] The invention furthermore concems an arrangement for coding live images in 
1 5 microscopy. In particular, the invention concems a system that improves, in terms of 
transmitted image quality, the transmission of microscope images fi-om a microscope to a 
remote station. 



20 [0004] In existing video coding as presently practiced, algorithms are used inside the 
codec to recognize image changes, in order to find the image portions that are to be 
compressed. The calculation time needed to discover such image changes (a person's 
head has moved, etc.) is relatively long and, together with the transmission bandwidth, 
limits the number of moving images that can be processed per second. 

25 [0005] US Patent 5,216,596 discloses a telepathology system. A workstation is set 
up at a remote location and receives images from a preparation (tissue) that is to be 
examined with a microscope. The microscope images are recorded with a conventional 
video camera, and displayed at the remote location on a conventional video monitor. A 
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digitization or coding of the image data is accomplished after imaging. The system 
presented here is tied to analog transmission links, and cannot achieve the necessary 
resolution in a conventional digital network. Coding is also ruled out because of the 
analog transmission. 



[0006] It is an object of the invention to create a method with which an increase in 
video image rate and a reduction in compression outlay can be achieved in the context of 
the transmission of live microscope images over digital networks. The method is also 

10 intended to make it possible to enhance display quality, i.e. to prevent flicker effects due 
to continuous image transmission even when images are stationary. 
[0007] According to the present invention, this is achieved by a method that 
comprises the following steps: 

a) recording a first complete image that depicts a portion of a microscopic 

15 preparation; 



20 preceding complete image in a plane defined by an X-Y stage; 

f) transferring the coordinates of the portion of the second complete image, 
and further control data, to a control data decoder; 

g) generating at least one coded partial image utilizing the data fi-om the 
control data decoder; 

25 h) generating an assembled and coded complete image in an image 

assembler, using the at least one coded partial image and the preceding coded complete 
image located in the buffer memory; 
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SUMMARY OF THE INVENTION 



b) 
c) 
d) 
e) 



generating a first coded complete image in a coding element; 
storing the first coded complete image in a buffer memory; 
outputting the first coded complete image; 

recording a second complete image that is offset with respect to the 
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i) outputting a second assembled and coded complete image, the assembled 



and coded complete image also being stored in the buffer memory for that purpose; and 



further image. 

5 [0008] A further object of the invention is to create an arrangement which makes 
possible flicker- free image transmission of live microscope images at an increased image 
rate. 

[0009] According to the present invention, this is achieved by an arrangement which 
comprises a coder to which complete images can be transferred, the coder comprising a 
10 coding unit that is connected to a buffer memory; a control data decoder being connected 
to the coding unit, to the buffer memory, and to an image assembler; and the image 
assembler receiving data from the buffer memory and transferring data to the buffer 
memory. 

[0010] One advantage of the invention is an increase in the video image rate and a 
1 5 reduction in compression outlay in the transmission of live microscope images over 
digital networks. In addition, display quality is enhanced, i.e. flicker effects due to 
continuous image transmission even when images are stationary, or flickering due to 
continuous compression, are prevented. Compression of the image data is accomplished 
with several commercially common algorithms, in order to adhere to existing video 
20 compression standards. 

[0011] A further advantage of the invention is that the coder/decoder algorithm 
^ according to the present invention takes into accoimt the circumstance that the status data 
of an automatic microscope are utilized for video coding of moving images or live 
images from a microscope. In microscopic examinations, no unexpected movements 
25 occur within the image. The movements of the specimen or preparation are usually only 
displacements in the three spatial coordinates X, Y, and Z. Using this additional 
information, the coding time for the image that is to be transmitted can be considerably 
shortened, data volume is reduced, and a higher moving-image rate or better image 



j) recording further images, steps f) through i) being repeated for each 
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quality (due to less-severe compression) is achieved. The algorithm used here utilizes 
additional input data; i.e. one control channel and two data channels with additional 
information are additionally used, as well as the image data, as input for the coder. The 
control channel contains information for controlling the partial coding of the input image. 
5 The data required for this (e.g. X-Y position, color values, etc.) are conveyed via the two 
additional data channels. 



BRIEF DESCRIPTION OF THE DRAWINGS 
[0012] The drawings schematically depict the subject matter of the invention, which 
10 is described below with reference to the Figures. In the drawings: 

FIG. 1 schematically depicts a system in which the invention is applied; 

FIG. 2 schematically depicts the recording of an image of a preparation, the position 

of the X-Y stage having been modified in the X position; 
FIG. 3 schematically depicts the recording of an image of a preparation, the position 
15 of the X-Y stage having been modified in the Y position; 

FIG. 4 schematically depicts the recording of an image of a preparation, the position 

of the X-Y stage having been modified in the X and Y positions; 
FIG. 5 shows a schematic configuration of a coder; and 
FIG. 6 shows a schematic configuration of a decoder. 

20 

DETAILED DESCRIPTION OF THE INVENTION 
[0013] System 1 depicted in FIG. 1 comprises a microscope 2 that is set up at a 
location where preparations (tissue sections) for examination are being produced. The 
location is usually a histology or pathology department of a hospital. A first computer 4 
25 having a monitor 5 is associated with microscope 2. A second computer 6, also having a 
monitor 7, is connected to first computer 4 via a conventional network 8. The network is 
represented in FIG. 1 by a connecting line having an interruption 8a, the better to 
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illustrate that second computer 6 can in principle be installed at any desired distance from 
first computer 4. 

[0014] In the exemplary embodiment depicted here, microscope 2 is depicted as an 
automatic microscope. All the imaging parameters of microscope 2 can be set and 
5 modified, for example, from second computer 6. In this case microscope 2 possesses 
corresponding motors for setting the parameters. FIG. 1 depicts a motor 10 that makes 
possible displacement of an X-Y stage 12. Motors for changing the imaging scale, 
moving X-Y stage 12 in the Z direction, and/or focusing are not depicted for reasons of 
clarity. The arrangement and use of these motors is, however, evident to one skilled in 

10 the art. First computer 4 usually serves to record the image data from microscope 2 and 
convert it into a corresponding data format for transfer via network 8. A compression of 
the image data can also be performed by first computer 4. In addition, microscope 2 is 
also equipped with position sensors (not depicted), which supply to first computer 4 
signals which provide information about the X, Y, and Z position of X-Y stage 12. It is 

15 also conceivable for X-Y stage 12, or individual components of the microscope, to supply 
signals which allow a position determination. First computer 4 is also responsible for 
receiving data from second computer 6 in order to control microscope 2, and for 
converting them into corresponding control signals. First and second computers 4 and 6 
are used for communication via network 8, "communication" being understood to mean 

20 data transfer in both directions. 

[0015] A specimen slide 14, with a preparation 14a on it, is placed on X-Y stage 12. 
Depending on the selected magnification, an image window (not depicted) is imaged and 
is recorded by a camera 16. Camera 16 can be configured, for example, as a 
conventional video camera or CCD camera. Camera 16 is connected via a connection 17 

25 to first computer 4. A fiirther connection 18, over which control signals are sent to the 
corresponding motors, exists between first computer 4 and microscope 2. Control signals 
are conveyed via network 8 from second computer 6 to first computer 4. Image data are 
similarly conveyed via network 8 from first computer 4 to second computer 6. The two 
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computers 4 and 6 can each be equipped with a WAN module 1 1 (ISDN, ASDL, ATM, 
satellite) that serves to establish a connection. An input unit 20 for user inputs is also 
connected to each computer 4 and 6. Input unit 20 can be configured as a mouse, 
keyboard, or voice control unit. 
5 [0016] FIGS. 2 through 4 illustrate the imaging of a portion of a tissue section 100. 
A first image 102 is recorded by a video camera or CCD camera. The camera defines a 
first image fi^ame 104 that is depicted in FIGS. 2 through 4 with solid lines. X-Y stage 12 
(see FIG. 1) is displaced in the X direction, and this results in an offset image fi-ame 106 
that is depicted in FIG. 2 with dashed lines. The difference between first image firame 
10 104 and offset image firame 106 is a cross-hatched area 108. 

[0017] In FIG. 3, the X-Y stage is displaced in Y direction Y, and this results in an 
offset image fi-ame 106 that in FIG. 3 is again depicted with dashed lines. The difference 
between first image fi-ame 104 and offset image fi-ame 106 is again a cross-hatched area 
108. 

15 [0018] In FIG. 4, the X-Y stage is displaced in X direction X and in Y direction Y, 
and this results in an offset image firame 106 that in FIG. 4 is again depicted with dashed 
lines. The difference between first image fi-ame 104 and offset image fi-ame 106 is a 
cross-hatched area 108. 

[0019] The arrangement must fiarthermore make a comparison to determine whether 
20 the recorded image has experienced any change in the Z direction. It is then also 

necessary to detect any changes in the image content in which the new image 

encompasses a region that is completely outside the region of the preceding image. 

Suitable processing and identification methods are available for this purpose. 

[0020] In order to improve the transmission of the recorded microscope images to a 
25 remote station and to increase the transmission speed, it is sufficient to transmit only the 

portion of the image that results fi-om the offset by X-Y stage 12. As is apparent from 

FIGS. 2 through 4, only cross-hatched area 108 of offset image frame 106 needs to be 

transmitted to yield a complete image at the receiving end. 
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[0021] FIG. 5 depicts a schematic configuration of a coder 21 for preparing for image 
transmission. At the beginning of the coding process, the type of image output must be 
defined. In this exemplary embodiment, coder 21 possesses three outputs. A coded 
complete image 200 that is completely coded is output at a first output 22]. A coded 
5 partial image 220 that is partially coded is output at a second output 222. An assembled 
and coded complete image 210 that comprises several partial images 210i and 2IO2 is 
output at third output 223. Each of the partial images 210i and 2IO2 is partially coded, 
and is correspondingly assembled into a coded complete image 210. Coder 21 
fiirthermore possesses four inputs. The image data of the input image (a first complete 

10 image 25 1) are transferred to coder 21 via a first input 24i. First complete image 25 1 can, 
for example, be recorded by way of a video camera or CCD camera (not depicted). 
[0022] The first recorded complete image 25 1 is transferred to a coding element 26 
and is always completely coded. The coded image is then stored in a buffer memory 27. 
A complete image can be output at first output 22 1. If, for example, X-Y stage 12 is 

15 displaced, a second complete image 252 is again recorded. In the example described 
here, the difference between the second recorded complete image 252 and the first 
recorded complete image 25 1 is a cross-hatched area 240. The second recorded complete 
image 252 is again conveyed to coding element 26. The procedure is the same for all 
fiirther recorded images. 

20 [0023] Coder 21 possesses a control data decoder 30 that has three inputs. A first 
input 26] is connected to a control channel 28 that supplies information for controlling 
partial coding of the input image. A first data channel 30i is connected to a second input 
262, and a second data channel 3O2 to a third input 263. First and second data channels 
30i and 3O2 supply, for example, information about the X-Y position of the X-Y stage, 

25 color values, or the like. Images that are recorded after the first recorded complete image 
25 1 are partially or completely coded as a function of the information firom control data 
decoder 30. 
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[0024] Coder 21 also possesses an image assembler 32 which also receives 
information from control data decoder 30 in order to assemble the coded partial images 
210i and 2IO2 into a coded complete image 210. Assembly of coded partial images 210i 
and 2IO2 into a complete image 210 is necessary when, as depicted in FIG. 5, the 
5 difference between the first recorded complete image 25 1 and the subsequently recorded 
complete image 252 is, for example, cross-hatched area 240. A datimi is also sent from 
control data decoder 26 to buffer memory 27 so that the image information necessary for 
assembly of a complete image is sent to image assembler 32. Once a complete image 
210 has been generated from the coded partial images 210i and 2IO2, it can be output via 
10 third output 223. In addition, the newly generated complete image 210 is stored in buffer 
memory 27 and thus constitutes a basis for possible assembly of a newly recorded input 
image. 

[0025] In addition to the pure image data, a coded complete image 210 or partial 
image 220 additionally contains information about the type of coding (complete/partial) 
15 and, in the case of partially coded images, information about the location of the image in 
the overall image. 

[0026] FIG. 6 shows a schematic configuration of a decoder 40. As already 
mentioned above, the input images for decoder 40 are the coded complete image 200 or 
coded partial image 220 of FIG. 5. These images also contain, in addition to the pure 
20 image information, information about the type of coding (complete image 200/partial 

image 220). In the case of the partially coded images, data conceming the position of the 
partial image in the overall image are additionally analyzed as input data. As aheady 
mentioned with reference to coding, the first image that is transmitted is a complete 
image 200. 

25 [0027] Decoder 40 possesses a control data decoder 42 that ascertains the 

corresponding position data and/or control data from complete image 200 or partial 
image 220 that is received. As aheady mentioned with regard to FIG. 5, the first image 
recorded is always a complete image 200, which is output from coder 21 as coded 
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complete image 200. A corresponding procedure is used in decoder 40. Control data 
decoder 42 receives the coded complete image 200 and forwards it to a decoder unit 44. 
The coded complete image 200 is converted into a decoded complete image 45 1 and 
output. The decoded complete image 45 1 corresponds to the first recorded complete 
5 image 25 1 before coding. The first decoded complete image 45 1 is additionally stored in 
decoder buffer memory 46. The procedure is the same for all fiirther complete images: 
decode, buffer memory, output. 

[0028] Coded partial images 220 are also transferred to control data decoder 42 and 
decoded, and then, as a fimction of the additional data (position in the overall image, 

10 etc.), combined in an image assembler 48 with the preceding image to form a decoded 
complete image 452. This complete image 452 is stored in decoder buffer memory 46 as 
the new preceding image, and additionally output. Monitor 5, 7 respectively associated 
with first or second computer 4, 6 is usually used as the output medium. 
[0029] The invention has been described with reference to one particular 

15 embodiment. It is self-evident, however, that changes and modifications can be made 
without thereby leaving the scope of protection of the claims recited hereinafter. 
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